Accuracy Of Machine Learning Techniques Research Articles

Classification of soybean groups for grain yield and industrial traits using Vnir-Swir spectroscopy

This research aimed to evaluate the accuracy of machine learning techniques in distinguishing groups soybean genotypes according to grain industrial traits using hyperspectral reflectance of the leaves. A total of 32 soybean genotypes were evaluated and allocated in randomized blocks with four replications. At 60 days after emergence, spectral analysis was carried out on three leaf samples from each plot. The spectral analysis of the leaves was carried out with a hyperspectral sensor providing ranges from 350 to 2500 nm. Once the wavelengths were obtained, they were grouped into averages of representative intervals into bands. At the end of the crop cycle, grain yield was obtained, and subsequently the determination of carbohydrate, oil, and protein content. Initially, the genotypes were subjected to cluster analysis using the k-means algorithm and subsequently, the data was subjected to machine learning analysis, using six models: J48 Decision Trees (J48) and REPTree (DT), Random Forest (RF), Artificial Neural Networks (ANW), Logistic Regression (LR) and Support Vector Machine (SVM). Logistic regression (LR) was used as a reference point as it is a traditional regression algorithm. The clusters formed acted as the output of the models, while for the input of the models, two groups of data were used: the spectral variables (SV) obtained by the sensor (350–2500 nm) and the spectral averages of the bands selected (BS) (350–2200 nm). The use of machine learning techniques presented lower responses than the standard technique used in the work, that is, LR, which presented superiority in the classification of soybean genotypes in terms of industrial traits. The use of wavelengths provided better performance of the algorithms in the classification in relation to selected bands.

Quantifying wildfire drivers and predictability in boreal peatlands using a two-step error-correcting machine learning framework in TeFire v1.0

Abstract. Wildfires are becoming an increasing challenge to the sustainability of boreal peatland (BP) ecosystems and can alter the stability of boreal carbon storage. However, predicting the occurrence of rare and extreme BP fires proves to be challenging, and gaining a quantitative understanding of the factors, both natural and anthropogenic, inducing BP fires remains elusive. Here, we quantified the predictability of BP fires and their primary controlling factors from 1997 to 2015 using a two-step correcting machine learning (ML) framework that combines multiple ML classifiers, regression models, and an error-correcting technique. We found that (1) the adopted oversampling algorithm effectively addressed the unbalanced data and improved the recall rate by 26.88 %–48.62 % when using multiple datasets, and the error-correcting technique tackled the overestimation of fire sizes during fire seasons; (2) nonparametric models outperformed parametric models in predicting fire occurrences, and the random forest machine learning model performed the best, with the area under the receiver operating characteristic curve ranging from 0.83 to 0.93 across multiple fire datasets; and (3) four sets of factor-control simulations consistently indicated the dominant role of temperature, air dryness, and climate extreme (i.e., frost) for boreal peatland fires, overriding the effects of precipitation, wind speed, and human activities. Our findings demonstrate the efficiency and accuracy of ML techniques in predicting rare and extreme fire events and disentangle the primary factors determining BP fires, which are critical for predicting future fire risks under climate change.

Comparing accuracy of machine learning approaches to identifying parathyroid adenomas: Lessons and new directions

PurposeThe purpose of this investigation is to understand the accuracy of machine learning techniques to detect biopsy-proven adenomas from similar appearing lymph nodes and factors that influence accuracy by comparing support vector machine (SVM) and bidirectional Long short-term memory (Bi-LSTM) analyses. This will provide greater insight into how these tools could integrate multidimensional data and aid the detection of parathyroid adenomas consistently and accurately. MethodsNinety-nine patients were identified; 93 4D-CTs of patients with pathology-proven parathyroid adenomas were reviewed; 94 parathyroid adenomas and 112 lymph nodes were analyzed. A 2D slice through the lesions in each phase was used to perform sequence classification with ResNet50 as the pre-trained network to construct the Bi-LSTM model, and the mean enhancement curves were used to form an SVM model. The model characteristics and accuracy were calculated for the training and validation data sets. ResultsOn the training data, the area under the curve (AUC) of the Bi-LSTM was 0.99, while the SVM was 0.95 and statistically significant on the DeLong test. The overall accuracy of the Bi-LSTM on the validation data set was 92 %, while the SVM was 88 %. The accuracy for parathyroid adenomas specifically was 93 % for the Bi-LSTM and 83 % for the SVM model. ConclusionEnhancement characteristics are a distinguishing feature that accurately identifies parathyroid adenomas alone. The Bi-LSTM performs statistically better in identifying parathyroid adenomas than the SVM analysis when using both morphologic and enhancement information to distinguish between parathyroid adenomas and lymph nodes. Summary statementThe Bi-LSTM more accurately identifies parathyroid adenomas than the SVM analysis, which uses both morphologic and enhancement information to distinguish between parathyroid adenomas and lymph nodes, performs statistically better.

Interpretable machine learning for imbalanced credit scoring datasets

The class imbalance problem is common in the credit scoring domain, as the number of defaulters is usually much less than the number of non-defaulters. To date, research on investigating the class imbalance problem has mainly focused on indicating and reducing the adverse effect of the class imbalance on the predictive accuracy of machine learning techniques, while the impact of that on machine learning interpretability has never been studied in the literature. This paper fills this gap by analysing how the stability of Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP), two popular interpretation methods, are affected by class imbalance. Our experiments use 2016–2020 UK residential mortgage data collected from European Datawarehouse. We evaluate the stability of LIME and SHAP on datasets of progressively increased class imbalance. The results show that interpretations generated from LIME and SHAP are less stable as the class imbalance increases, which indicates that the class imbalance does have an adverse effect on machine learning interpretability. To check the robustness of our outcomes, we also analyse two open-source credit scoring datasets and we obtain similar results.

Financial reporting quality and its determinants: A machine learning approach

The high-quality of financial reporting provides suitable information for economic decision-making of the country whilst, the low quality of financial reporting causes a serious impact on the economy. This research aims to classify financial reporting quality (FRQ) as well as determines the drivers of FRQ. This study uses a panel dataset from 2014 to 2020 that is collected from the Vietnamese listed companies. The study applies machine learning algorithms to classify and assess FRQ of non-financial companies on the Vietnamese stock exchange. New contribution considers the FRQ, on the auditor's opinion and the variance between pre-audit and post-audit profit. This research classifies FRQ into normal and poor categories, and a rate of 9.35% in the sample is considered poor FRQ. This research shows that the return on assets’ ratio and the ownership concentration have the most important influence on FRQ. Furthermore, the results which are predicting FRQ by using the random forest algorithm have an accuracy rate of 94%. This study is valuable for the forecast of FRQ and for the support of stakeholders in decision-making. With the high accuracy of machine learning techniques and its usage, it can help analysts and investors in generating reliable accounting information for decision-making purposes. Corporate sector needs to pay attention towards financial ratios and reinforcement of corporate governance.

Impact of feature reduction techniques on classification accuracy of machine learning techniques in leg rehabilitation

The scientific community has recently shown a lot of interest in biometric-based person recognition systems. It is a dynamic technology that tries to recognize biometrics automatically, swiftly, precisely, and consistently. Gait recognition is a sort of biometric classification that concentrates on identifying persons utilizing personal measurements and correlations, including trunk and limb size, in addition to spatial data connected to innate patterns in people's motions. The purpose of this study is to highlight how important it is to use feature reduction strategies to increase classification accuracy. Two of these approaches are employed in this work: Principal Component Analysis (PCA) and Singular Value Decomposition (SVD). To classify foot disease, six machine learning techniques are deployed as classifiers. These classifiers are Random Forest (RF), Naïve Bayes (NB), K-Nearest Neighbors (KNN), Logistic Regression (LR), Decision Tree (DT), and Stochastic Gradient Descent (SGD) for the purpose of determining which classifier performs best in classifying leg rehabilitation data. Experimental results using the EMG dataset in Lower Limb indicate that the classification accuracy reached 99% with a time not exceeding a few seconds.

A systematic quantitative review on the performance of some of the recent short-term rainfall forecasting techniques

Abstract Rainfall forecasting is a high-priority research problem due to the complex interplay of multiple factors. Despite extensive studies, a systematic quantitative review of recent developments in rainfall forecasting is lacking in the literature. This study conducted a systematic quantitative review of statistical, numerical weather prediction (NWP) and machine learning (ML) techniques for rainfall forecasting. The review adopted the preferred reporting items for systematic reviews and meta-analyses (PRISMA) technique for screening keywords and abstracts, leading to 110 qualified papers from multiple databases. The impact of rainfall threshold, meteorological parameters, topography, algorithm techniques, geographic location, the horizontal resolution of the model, and lead time on rainfall forecast was examined. The review shows the importance of precipitable water vapor (PWV) along with other meteorological parameters for accurate nowcasting in coastal and mountainous regions. An increase in rainfall forecast uncertainty with an increase in the lead time makes the NWP model less popular for the short-term forecast. The pre-processing techniques increased the accuracy of ML techniques by considering extreme values and detecting the irregularly distributed multi-scale features of rainfall in space and time. Future research can focus on hybrid models with improved accuracy for nowcasting. The output from the hybrid model serves as input for the decision support system required for urban flood risk management.

Investigation of fault detection and isolation accuracy of different Machine learning techniques with different data processing methods for gas turbine

Classification is an essential task for many applications, including text classification, image classification, data classification, and so on. The present study investigates the accuracy of different machine learning classification algorithms with three different data smoothing techniques for gas turbine fault detection and isolation task. The gas turbine performance model was developed by considering variable inlet guide vane and bleed air. Fouling and erosion were injected into all six main components of the gas turbine engine. Faulty and non-faulty data were generated from the developed performance model. Based on sensitivity analysis, 12 measurement parameters and 11,824 data points were selected for the development of a fault detection and isolation model. The faulty and non-faulty data were balanced, smoothed, corrected and normalized. Finally, the classification accuracy of the machine learning techniques was analyzed. The result shows that K-Nearest Neighbours, Neural Network and Decision Tree classifiers exhibited high classification accuracy, about 99% with all three data smoothing techniques. It is also observed that the computation time of Support Vector Machine is higher whereas K-Nearest Neighbours shows the lowest. Finally, the research proves that K-Nearest Neighbours is the best classification technique for gas turbine engine fault detection and isolation application.

Economic policy uncertainty and bankruptcy filings

Applying machine learning techniques to predict bankruptcy in the sample of French, Italian, Russian and Spanish firms, the study demonstrates that the inclusion of economic policy uncertainty (EPU) indicator into bankruptcy prediction models notably increases their accuracy. This effect is more pronounced when we use novel Twitter-based version of EPU index instead of original news-based index. We further compare the prediction accuracy of machine learning techniques and conclude that stacking ensemble method outperforms (though marginally) machine learning methods, which are more commonly used for bankruptcy prediction, such as single classifiers and bagging.

Gradient boosting-based regression modelling for estimating the time period of the irregular precast concrete structural system with cross bracing

Regression tree, random forest, bagging and gradient boosting regression-based modelling techniques were used to estimate the time period of precast concrete frame structures with vertical irregularity and cross-bracing using 756 Etabs models. This paper thoroughly explored the efficacy of random forest, regression tree, bagging, gradient boosting-based modelling using RStudio. The time period has been the output parameter while the number of cross-bracing, column size, beam size, soft storey, irregularity coefficient and height of the building has been assigned as input parameters. The accuracy of machine learning techniques has been checked by reference to the formulas published in the literature. A comparison of results indicates that the gradient boosting-based regression approach performed well as compared to random forest regression, bagging and regression tree.

Comparison of Outdoor Compost Pile Detection Using Unmanned Aerial Vehicle Images and Various Machine Learning Techniques

Since outdoor compost piles (OCPs) contain large amounts of nitrogen and phosphorus, they act as a major pollutant that deteriorates water quality, such as eutrophication and green algae, when the OCPs enter the river during rainfall. In South Korea, OCPs are frequently used, but there is a limitation that a lot of manpower and budget are consumed to investigate the current situation, so it is necessary to efficiently investigate the OCPs. This study compared the accuracy of various machine learning techniques for the efficient detection and management of outdoor compost piles (OCPs), a non-point pollution source in agricultural areas in South Korea, using unmanned aerial vehicle (UAV) images. RGB, multispectral, and thermal infrared UAV images were taken in August and October 2019. Additionally, vegetation indices (NDVI, NDRE, ENDVI, and GNDVI) and surface temperature were also considered. Four machine learning techniques, including support vector machine (SVM), decision tree (DT), random forest (RF), and k-NN, were implemented, and the machine learning technique with the highest accuracy was identified by adjusting several variables. The accuracy of all machine learning techniques was very high, reaching values of up to 0.96. Particularly, the accuracy of the RF method with the number of estimators set to 10 was highest, reaching 0.989 in August and 0.987 in October. The proposed method allows for the prediction of OCP location and area over large regions, thereby foregoing the need for OCP field measurements. Therefore, our findings provide highly useful data for the improvement of OCP management strategies and water quality.

The Role of Data Pre-processing Techniques in Improving Machine Learning Accuracy for Predicting Coronary Heart Disease

These days, in light of the rapid developments, people work day and night to live at a good level. This often causes them to not pay much attention to a healthy lifestyle, such as what they eat or even what physical activities they do. These people are often the most likely to suffer from coronary heart disease. The heart is a small organ responsible for pumping oxygen-rich blood to the rest of the human body through the coronary arteries. Accordingly, any blockage or narrowing in one of these coronary arteries may cause blood not to be pumped to the heart and from it to the rest of the body, and thus cause what is known as heart attacks. From here, the importance of early prediction of coronary heart disease has emerged, as it can help these people change their lifestyle and eating habits to become healthier and thus prevent coronary heart disease and avoid death. This paper improve the accuracy of machine learning techniques in predicting coronary heart disease using data preprocessing techniques. Data preprocessing is a technique used to improve the efficiency of a machine learning model by improving the quality of the feature. The popular Framingham Heart Study dataset was used for validation purposes. The results of the research paper indicate that the use of data preprocessing techniques had a role in improving the predictive accuracy of poorly efficient classifiers, and shows satisfactory performance in determining the risk of coronary heart disease. For example, the Decision Tree classifier led to a predictive accuracy of coronary heart disease of 91.39% with an increase of 1.39% over the previous work, the Random Forest classifier led to a predictive accuracy of 92.80% with an increase of 2.7% over the previous work, the K-Nearest Neighbor classifier led to a predictive accuracy of 92.68% with an increase of 2.58% over the previous work, the Multilayer Perceptron Neural Network (MLP) classifier led to a predictive accuracy of 92.64% with an increase of 2.64% over the previous work, and the Na¨ıve Bayes classifier led to a predictive accuracy of 90.56% with an increase of 0.66% over the previous work.

Machine Learning Techniques for Software Maintainability Prediction: Accuracy Analysis

Maintaining software once implemented on the end-user side is laborious and, over its lifetime, is most often considerably more expensive than the initial software development. The prediction of software maintainability has emerged as an important research topic to address industry expectations for reducing costs, in particular, maintenance costs. Researchers and practitioners have been working on proposing and identifying a variety of techniques ranging from statistical to machine learning (ML) for better prediction of software maintainability. This review has been carried out to analyze the empirical evidence on the accuracy of software product maintainability prediction (SPMP) using ML techniques. This paper analyzes and discusses the findings of 77 selected studies published from 2000 to 2018 according to the following criteria: maintainability prediction techniques, validation methods, accuracy criteria, overall accuracy of ML techniques, and the techniques offering the best performance. The review process followed the well-known systematic review process. The results show that ML techniques are frequently used in predicting maintainability. In particular, artificial neural network (ANN), support vector machine/regression (SVM/R), regression & decision trees (DT), and fuzzy & neuro fuzzy (FNF) techniques are more accurate in terms of PRED and MMRE. The N-fold and leave-one-out cross-validation methods, and the MMRE and PRED accuracy criteria are frequently used in empirical studies. In general, ML techniques outperformed non-machine learning techniques, e.g., regression analysis (RA) techniques, while FNF outperformed SVM/R, DT, and ANN in most experiments. However, while many techniques were reported superior, no specific one can be identified as the best.

Soil type spatial prediction from Random Forest: different training datasets, transferability, accuracy and uncertainty assessment

ABSTRACT: Different uses of soil legacy data such as training dataset as well as the selection of soil environmental covariables could drive the accuracy of machine learning techniques. Thus, this study evaluated the ability of the Random Forest algorithm to predict soil classes from different training datasets and extrapolate such information to a similar area. The following training datasets were extracted from legacy data: a) point data composed of 53 soil samples; b) 30 m buffer around the soil samples, and soil map polygons excluding: c) 20 m; and d) 30 m from the boundaries of polygons. These four datasets were submitted to principal component analysis (PCA) to reduce multidimensionality. Each dataset derived a new one. Different combinations of predictor variables were tested. A total of 52 models were evaluated by means of error of models, prediction uncertainty and external validation for overall accuracy and Kappa index. The best result was obtained by reducing the number of predictors with the PCA along with information from the buffer around the points. Although Random Forest has been considered a robust spatial predictor model, it was clear it is sensitive to different strategies of selecting training dataset. Effort was necessary to find the best training dataset for achieving a suitable level of accuracy of spatial prediction. To identify a specific dataset seems to be better than using a great number of variables or a large volume of training data. The efforts made allowed for the accurate acquisition of a mapped area 15.5 times larger than the reference area.

Analysing the accuracy of machine learning techniques to develop an integrated influent time series model: case study of a sewage treatment plant, Malaysia.

The function of a sewage treatment plant is to treat the sewage to acceptable standards before being discharged into the receiving waters. To design and operate such plants, it is necessary to measure and predict the influent flow rate. In this research, the influent flow rate of a sewage treatment plant (STP) was modelled and predicted by autoregressive integrated moving average (ARIMA), nonlinear autoregressive network (NAR) and support vector machine (SVM) regression time series algorithms. To evaluate the models' accuracy, the root mean square error (RMSE) and coefficient of determination (R2) were calculated as initial assessment measures, while relative error (RE), peak flow criterion (PFC) and low flow criterion (LFC) were calculated as final evaluation measures to demonstrate the detailed accuracy of the selected models. An integrated model was developed based on the individual models' prediction ability for low, average and peak flow. An initial assessment of the results showed that the ARIMA model was the least accurate and the NAR model was the most accurate. The RE results also prove that the SVM model's frequency of errors above 10% or below - 10% was greater than the NAR model's. The influent was also forecasted up to 44weeks ahead by both models. The graphical results indicate that the NAR model made better predictions than the SVM model. The final evaluation of NAR and SVM demonstrated that SVM made better predictions at peak flow and NAR fit well for low and average inflow ranges. The integrated model developed includes the NAR model for low and average influent and the SVM model for peak inflow.

Phase segmentation of X-ray computer tomography rock images using machine learning techniques: an accuracy and performance study

Abstract. Performance and accuracy of machine learning techniques to segment rock grains, matrix and pore voxels from a 3-D volume of X-ray tomographic (XCT) grayscale rock images was evaluated. The segmentation and classification capability of unsupervised (k-means, fuzzy c-means, self-organized maps), supervised (artificial neural networks, least-squares support vector machines) and ensemble classifiers (bragging and boosting) were tested using XCT images of andesite volcanic rock, Berea sandstone, Rotliegend sandstone and a synthetic sample. The averaged porosity obtained for andesite (15.8 ± 2.5 %), Berea sandstone (16.3 ± 2.6 %), Rotliegend sandstone (13.4 ± 7.4 %) and the synthetic sample (48.3 ± 13.3 %) is in very good agreement with the respective laboratory measurement data and varies by a factor of 0.2. The k-means algorithm is the fastest of all machine learning algorithms, whereas a least-squares support vector machine is the most computationally expensive. Metrics entropy, purity, mean square root error, receiver operational characteristic curve and 10 K-fold cross-validation were used to determine the accuracy of unsupervised, supervised and ensemble classifier techniques. In general, the accuracy was found to be largely affected by the feature vector selection scheme. As it is always a trade-off between performance and accuracy, it is difficult to isolate one particular machine learning algorithm which is best suited for the complex phase segmentation problem. Therefore, our investigation provides parameters that can help in selecting the appropriate machine learning techniques for phase segmentation.

Forecasting Financial Time Series with Grammar‐Guided Feature Generation

The application of machine learning techniques to forecast financial time series is not a recent development, yet it continues to attract considerable attention because of the difficulty of the problem that is compounded by the nonlinear and nonstationary nature of the time series. The choice of an appropriate set of features is crucial to improve forecasting accuracy of machine learning techniques. In this article, we propose a systematic way for generating rich features using context‐free grammars. Our proposed methodology identifies potential candidates for new technical indicators that consistently improve forecasts compared with some well‐known indicators. The notion of grammar families as a compact representation to generate a rich class of features is exploited, and implementation issues are discussed in detail. The proposed methodology is tested on closing price data of major stock market indices, and the forecasting performance is compared with some standard techniques. A comparison with the conventional approach using standard technical indicators and naive approaches is shown.

The Combined Effect of Applying Feature Selection and Parameter Optimization on Machine Learning Techniques for Solar Power Prediction

This paper empirically shows that the combined effect of applying the selected feature subsets and optimized parameters on machine learning techniques significantly improves the accuracy for solar power prediction. To provide evidence, experiments are carried on in two phases. For all the experiments the machine learning techniques namely Least Median Square (LMS), Multilayer Perceptron (MLP) and Support Vector Machine (SVM) are used. In the first phase five well-known wrapper feature selection methods are used to obtain the prediction accuracy of machine learning techniques with selected feature subsets and default parameter settings. The experiments from the first phase demonstrate that holding the default parameters, LMS, MLP and SVM provides better prediction accuracy (i.e. reduced MAE and MASE) with selected feature subsets rather than without selected feature subsets. After getting improved prediction accuracy from the first phase, the second phase continues the experiments to optimize machine learning parameters and the prediction accuracy of those machine learning techniques are re-evaluated through adopting both the optimized parameter settings and selected feature subsets. The comparison between the results of two phases clearly shows that the later phase (i.e. machine learning techniques with selected feature subsets and optimized parameters) provides substantial improvement in the accuracy for solar power prediction than the earlier phase (i.e. machine learning techniques with selected feature subsets and default parameters). Experiments are carried out using reliable and real life historical meteorological data. The machine learning accuracy of solar radiation prediction is justified in terms of statistical error measurement and validation metrics. Experimental results of this paper facilitate to make a concrete verdict that providing more attention and effort towards the feature subset selection and machine learning parameter optimization (e.g. combined effect of selected feature subsets and optimized parameters on prediction accuracy which is investigated in this paper) can significantly contribute to improve the accuracy of solar power prediction.

The Effectiveness of Feature Selection Method in Solar Power Prediction

This paper empirically shows that the effect of applying selected feature subsets on machine learning techniques significantly improves the accuracy for solar power prediction. Experiments are performed using five well-known wrapper feature selection methods to obtain the solar power prediction accuracy of machine learning techniques with selected feature subsets. For all the experiments, the machine learning techniques, namely, least median square (LMS), multilayer perceptron (MLP), and support vector machine (SVM), are used. Afterwards, these results are compared with the solar power prediction accuracy of those same machine leaning techniques (i.e., LMS, MLP, and SVM) but without applying feature selection methods (WAFS). Experiments are carried out using reliable and real life historical meteorological data. The comparison between the results clearly shows that LMS, MLP, and SVM provide better prediction accuracy (i.e., reduced MAE and MASE) with selected feature subsets than without selected feature subsets. Experimental results of this paper facilitate to make a concrete verdict that providing more attention and effort towards the feature subset selection aspect (e.g., selected feature subsets on prediction accuracy which is investigated in this paper) can significantly contribute to improve the accuracy of solar power prediction.